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ABSTRACT 

The measurement of institutional effectiveness 
involves a systematic comparison of organizational purpose and 
performance. For community colleges, organizational purpose can be 
defined in terms of providing access to education, realizing student 
achievement, promoting student development, or addressing social 
needs. If all four purposes are acknowledged, then measures of 
effectiveness must be developed for each. At Macomb Community college 
(MCC) , an effort was made to determine the institution's main 
purposes, select appropriate goals for each purpose, and identify the 
specific measures which accurately indicate ; nstitutional outcomes. 
The process included a survey of students to cetermine their 
definition of a successful college experience; the formation of four 
study teams to develop lists of outcomes measures for successful 
transfer, completion of personal goals, employability , community 
impact, satisfaction indices, and human development; and the 
selection of five or six goals for each institutional purpose. The 
resulting statement of MCC's purposes and goals included the 
following: (l) provide opportunity through equitable access; 
effective services; social, cultural, and cross-cultural 
opportunities; student/client goal clarification; and student/client 
retention; (2) realize student achievement, as measured by completion 
of the student's goals, academic success, successful transfer, and 
successful employment; (3) promote student development in terms of 
learning and human potential; and (4) address societal needs, such as 
promoting quality of life and well-being, institutional innovation in 
problem solving, developing nhe community, and 

student/client/community satisfaction. Many of the specific measures 
designated for initial implementation already have data collection in 
progress. The next step is to establish a routine and a format for 
annual distribution of this information, and to increase awareness 
and understanding of the project. A chart illustrating institutional 
purposes and goals, and related measures, data sources, tracking 
frequency, and anticipated uses is attached. (JMC) 
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PHILOSOPHY 



Organizations that serve the public interest have the dual 
responsibility to "do things right" and to "do the right things." 
Although evaluative assessments have always been a major 
tool of organizations in planning and refining their activities, 
these efforts have traditionally only centered around the 
assessment of performance to improve processes and pro- 
cedures—to know if things are done right. However, it takes a 
broader evaluation to determine if the activities u ndertaken by 
the organization are the right things to do. This broader 
evaluation assesses the impact that an organization has on its 
constituency— whether or not its performance achieves the 
purposeforwhich it was created. Institutional effectiveness is 
measured by both performance and achievement of purpose. 

There has been substantial and growing interest over the 
last few years in the topic of measuring effectiveness of 
educational institutions, consistent with a general public 
sentiment for more accountability among public agencies of 
all types. More tnan ever, the public is looking to higher 
education to solve the eco nomic ills of th e cou ntry and to keep 
our businesses co mpetitiv ^ with foreign enterprises. And they 
want results. A recent study by the Michigan Department of 
Education (Project Outreach, 1989) has tracked the growing 
importance of education in this state. The study found that 88 
percent of our citizens say they feel a college education is 
"very important," up from 74 percent in 1 984. This new empha 
sis on education has brought about public outcry for reform 
and effectiveness of educational institutions. 

In some states, the measurement and documentation of 
institutional effectiveness has been recently legislated for 
higher education and tied to funding. In addition, it has also 
become a basis for accreditation. Even in areas where it has 
not been mandated, there has still been a great interest in 
evaluating effectiveness based upon this legislative trend and 
the recognition of its value, both programmatically and man- 
agerially. 

The topic of institutional effectiveness is of particular 
interestto community college leaders as they shift their focus 
from establishment and growth (as were their concerns through 
the 196C*3 and 1970*s) to quality and the utilization of re- 
sources in their organizations. It is quite natural for them to 
refieu on the accomplishment of their mission as the mstitu- 
tion becomes mature and growth stabilizes. 

Measurement of institut.onal effectiveness hasteencaiied 
"matching performance with purpose." Accordi ng to the South- 
ern Association of Colleges and Schools, 'The assessment of 
organizational effectiveness essentially involves a system- 
atic, explicit, and documented comparison of organizational 
performance to organizational purpose." The National Alii 
ance of Community and Technical Colleges (NACTC) calls 
such assessment a linkage between an organ\ation's mis- 
sion statement, the goals whicn it Implies, and the outcomes. 
NACTC developed a model to show this linkage. 
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As Peter Ewell has pointed out (Chronicle of Higher 
Education, 1 989). a college that is sencus about assessment 
should look at its mission and decide which educational 
outcomes fit that mission. Too often, the purpose of an 
educational institution is stated in vague terms in its mission 
statement. How you define purpose defines which outcomes 
you measure, that is, how you measure effectiveness. 

Tor community colleges, institutional purpooehasbeenan 
evolving concept. They were originally established tc provide 
greater access to higher education opportunities for those 
who had traditionally been excluded due to economic factors, 
ethnicity, race, gender, age, or even mobility. However, 
community colleges are now created and supported not only 
for the purpose of providing access and opportunities to 
community residents, but to serve their students and commu- 
nities in other ways as well. Community colleges now play an 
active role in helping individual students to realize their own 
educational goals and to maximize the development of their 
talents and skills. They are also instrumental in the develop- 
ment of communities and in so!.:ng the greater societal 
problems that their communities face. 

Providing opportunity is usually stressed in the establish- 
ment and growth stages of a community college. A new 
institution .nust emphasize delivery of their courses and 
services. If purpose is framed in terms of "opportunity," then 
effectiveness will be measured in terms of input factors, such 
as the number of students admitted and enrolled or the 
number of sections offered. 



Traditionally, selective colleges and universities have 
viewed their effectiveness as student academic achievement 
as portrayed by test scores or academic ho.iors. As noted by 
Alexander Astin, this is typically viewed without regard to prior 
academic preparation. For community colleges, the focus of 
achievement is relative to the completion of students* educa- 
tional goals. If purpose is stated in terms of "achievement," 
then effectiveness would be measured in terms of outcomes 
such as course and program completions, the number of 
degrees granted, or a companson of student status to student 
intent. 

A third alternative is to define purpose in terms of human 
development. It is the change which occurs in the individual as 
a result of having been involved ir. the learning process. 
Community colleges tend to actively assist this process with 
extensive learning support systems. If development is ac- 
cepted as an institutional purpose, then effectiveness would 
be measured in terms of improvement, such as increases in 
knowledge, skills and other intellectual and/or physical ca- 
pacities. 

Thef ou rth approach is to define pu rpose in terms of "social 
benefit" or rrr Ming "societal needs," that is, looking to the 
underlying contributions that the institution is expected to 
provide to society. If societal needs are selected for the 
statement of purpose, then effectiveness would be measured 
by how well those needs are being met by the actions of the 
institution. It requires that these measurements be made in 
the context of the perceptions of the community being served. 

When an organization enters the "maturity ,, stage — the 
stage where excellence rather than growth is used as the 
yardstick— greater interest isgiventothe achievement, devel- 
opment and societal needs aspects of their purpose. It is safe 
to assume that mature educational institutions probably de- 
sire to address all four pu rposes, although they will seldom be 
detailed in theirmissionstatements, since mission statements 
typically outline activities. If all four purposes are acknowl- 
edged, then effectiveness measures would be appropriate for 
each category. The challenge in front of us, then, is to select 
appropriate goals for each purpose and to identify the specific 
measures that serve as accurate indicators of the accomplish- 
ment of that purpose. 



PROCESS 

A as it was published and distributed in 1988, the 
Repc the Commission on The Future of Communily 
Colleges aroused a great deal of interest across the nation 
and at Macomb Community College in particular. Entitled 
"Building Communities: A Vision for A New Century," it 
generaled thinking about a number of topics essential to the 
future of our College. Among these was the challenge of 
assessing the outcomes of th e community college experience. 
The report stated, "We recommend that each community 
college develop a campus-wide assessment of institutional 
effectiveness. Such a program should include a periodic re- 
^ va minationof mission and goals, specificprograms.individual 
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student outcomes, retention rateb, and the performance of 
graduates." 

At Macomb's invitation, Terry O Banicn came to campus 
and addressed the members of the Colleges Manager's 
Croup. Through presentations and informal discussions, we 
explored the challenges of "Building Communities." 

January 1989 Faculty Development Day activities were 
structured with round table discussions to capture the ideas of 
faculty and professional staff relative to the topics of the report, 
including the need and impact of measuring and documenting 
institutional effectiveness, and related issues such as exiting 
competency testing and academic support systems. 

The College leadership explored alternative measures of 
success, shifting emphasisfromtraditionalmeasuresto those 
more appropriate for renewal. These include a shift . . . 

from quantity to quality 

from efficiency to effectiveness 

from inputs to outcomes 

from products to customers 

from repetition to innovation 

from discussion to action 

At this point, planning became even more focused on 
institutional renewal. The key elements of this renewal were 
identified as mission, leadership, culture/climate, staff devel- 
opment, strategic process, resource reallocation, and qualita- 
tive evaluation. The latter is expressed in terms of two visible 
goals, (1) institutional effectiveness and (?) student success. 
The College is committed to dealing with these goals overthe 
period of the next year. 

Discussions about renewal continued in context of a 
situational analysis of community colleges and the search for 
alternative measures of success. Approximately 600 students 
were surveyed to complete the sentence, "I would consider 
my experience at Macomb Community College a success 
if . . ." Responses revealed that students thought of their suc- 
cess in terms of both their educational experience while at 
Macomb and their future career or tranjfer success resulting 
from their work at Macomb. 

Four study teams were formed and each was asked to 
develop a list of 10 measures of the outcomes of institutional 
effectiveness, along with rationale as to their choices. The 
results of the study teams* efforts were completed in a wide 
variety of formats. The assumptions framing their reports 
included: 

"Institutional effectiveness is the process of articulating 
the mission of the college, setting goals, defining how 
the goals will be met, and using data from assessment 
for ongoing goal setting and planning." 

'There must be a fundamental shift in college function- 
ing from traditional to student-centered; goals and 
objectives must be achieved within available resources." 

"Because of their unique role in higher education, com- 
munity colleges must develop standards of excellence 
reflecting conformity to community needs. Because 
the College is basically a service organization ... its 
effectiveness should be judged on whether those serv- 
ices have made a positive difference in our students. 
As a human development institution, the development 



of human talent must be our principal goal. Conse- 
quently, effectiveness measures for the college must 
be 'outcomes' or Value added' in approach." 

With these introductory remarks, a total of 56 items were 
submitted. At first, these suggested measures appeared to 
cluster into common areas and a taxonomy of 

Successful Transfer 

Completion of Personal Goals 

Employability 

Community Impact 

Satisfaction Indices 

Human Development (Cultural/Social, 
Knowledge/Skill, and Intellectual) 
But not all measures fit into these categories, nor did they 
portray the varying importance of different measures. Other 
taxonomies were also explored, including those based upon 
audience 'students, employers, community), relationship to 
the institut on (institutional outcomes, external outcomes, and 
student outcomes); and institutional purpose. 

The approach of viewing effectiveness in context of insti- 
tutional purpose was well documented in the literature, and 
presented a logical and practical framework. The appropriate 
type of measure could then be determined based upon the 
purpose, before the task of identifying the specific effective- 
ness measurements was undertaken. 

STATED PURPOSE TYPE OF MEASURE 



Provide "Opportunity" 
Realize "Achievement" 
Promote "Development" 
Address "Societal Needs" 



Input factors/Quantitative 
Outcome Factors/Quaiitative 
Positive Change/Improvements 
Meeting Demands/Goals/Plans 



Goals that were identified for each institutional purpose 
were submitted by each study team ana compiled to form an 
institutional list of fifteen goals. Representatives from each 
study team were asked to narrow the list to a manageable five 
or six goals for initial implementation of measurement, and to 
identify the specific measures to be collected, documented 
and tracked over time. The specific measures were chosen 
based upon their face validity, practicality of collection, and 
utility of the data. 

The communication of the results of this process begins 
with this document, to be followed by the collection, reporting 
and distribution of identified measures. The full use of the data 
for continuous improvement is still a challenge in front us. 



PRODUCT 

Throughout this process, a great deal of introspection and 
clarification occurred. The results include a renewed commit- 
ment to assessment of institutional effectiveness, a set of 
goals identified by purpose, a specific list of initial measures 
to be tracked and documented, and a dedication to the use ol 
the data for institutional improvement. 

The renewed commitment is articulated in a Directional 
statement as adopted by the College. This Statement 
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clarifies the College's intent as it begins to assess institutional 
effectiveness. It also provides the parameters for action while 
allowing for flexibility in the implementation. 

Directional Goal: 

Assessment of Institutional Effectiveness 

Macomb Community College will undertake assess- 
ment of institutional effectiveness as part o f the plan- 
ning and renewal process. This assessment will com- 
pare performance with our evolving purpose, including 
(1) providing higher education opportunities, (2) realiz- 
ing academic achievement, (3) promoting human 
development, and (4) addressing societal needs. We 
commit to the systematic and ongoing measurement of 
relevant indicators and to the use of resulting informa- 
tion for continuous improvement. 

Within this general direction, the following goals are iden- 
tified as those most important to the achievement of our 
purposes: 

Provide "Opportunity" 
Equitable Access 
Effective Services 

Social, Cultural and Cross Cultural Opportunities 
Student/Client Goal Clarification 
Student/Client Retention 

Realize "Achievement" 
Student/Client Goal Completion 
Student/Client Academic Success 
Successful Transfer 
Successful Employment 

Promote "Development" 

Student/Client Learning 

Development of Human Potential 
Address "Societal Needs" 

Quality of Life and Well-Being 

Institutional Innovation in Problem Solving 

Development of Community 

Student/Client/Community Satisfaction 

Many of the specific measures designated for initial im- 
plementation already have data collection in progress. They 
are presented here with their sources and recommended 
frequency of reporting. While most of the measurements are 
under way for degree-credit programs, it will be necessary to 
expand the data collection to include non-credit students, 
those in Professional and Continuing Education (PACE) and 
the Center for Human Resource Development (CHRD). 

In addition to data collection and information distribution, 
the institutional commitment includes the use of effectiveness 
measures forcontinuous institutional improvement. The list of 
anticipated uses provides examples. It is expected that further 
uses can be identified and implemented by various depart- 
ments throughout the College. 

With the initial specific measures now identified, the next 
step in implementing the measurement of institutional effec- 
tiveness is to establish a routine and format for annua* 
distribution of this information. More critical, however, is 
meeting the continuing challenge of increasing awareness 
and understanding of this project and achieving its potential 
for institutional improvement. 



MACOMB COMMUNITY COLLEGE 
MEASURES OF INSTITUTIONAL EFFECTIVENESS RELATING TO PURPOSE 



INSTITUTIONAL 
PURPOSE & GOALS 



MEASURES 



DATA 
SOURCE 



TRACKING 
FREQUENCY 



ANTICIPATED USES 



PROVIDE "OPPORTUNITY" 

Equitable Access 



Effective Services 



Enrollment 
Headcount 



Student 
Satisfaction 



Effective Learning 
Support 



i REALIZE "ACHIEVEMENT" 

Successful Transfer 
Successful Employment 
Personal Interest 



Course 

Completion 

Rate 



Comparison 
of Student 
Status to 
Student Intent 



■ PROMOTE "DEVELOPMENT" 
Student/Client Learning Learning 

Outcomes 



MCC Data- Annual Market segmentation 
baseJPEOS, Target recruitment activities 

h.s. feeder rpts Estimate program vitality 



ACT Student Every 
Satisfaction 2 years 
Survey; PROE 



Match services with use and 

demand levels 
Determine staffing and operation needs 
Revise service offerings 
Co-curricular program development 
Reallocation of resources 



Grade Dist Semester Direct focus of academic alert system 
Reports Pi escription for short-term interventions 

Curriculum revisions 

Guide for course and program advising 



Registration 
Data;MiSIS; 
Follow-up 
Studies* 



CAAP* 



Annual Articulation 

Curriculum development 
Program and course development, 

revisions, and deletions 
Review of use of non-credit options 



Every Identify skills being taught 

2 years Identify topics for faculty development/ 

in-services 
Experimentation of teaching methods 
Reputation development 
Communication of identifiable results 

to the community 
Legislative correspondence to influence 

possible state mandates for outcomes 
NCA Accreditation 



I ADDRESS "SOCIETAL NEEDS" 

Client/Community 
Satisfaction 



Employer/ 
Community 
Satisfaction 



MiSIS/PROE/ Annual Strategic planning 
PREP, Curriculum development 

Telephone Changes in programs 

Surveys* Reallocation of resources 

Reputation enhancement 



'Not currently fully in Mace 
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